Mixed Reality–Based Slit Lamp for Ophthalmic Examination and Telemedicine: Technological Development and Validation Study

Introduction

The slit-lamp biomicroscope is a fundamental diagnostic tool in ophthalmology for detailed examination of both the anterior and posterior segments of the eye [1] with stereoscopic visualization [2]. Traditional slit lamps require the operator to view through the eyepieces to obtain a stereoscopic perspective. This necessity means the operator must stay close to the patient, which has become more concerning since the COVID-19 pandemic, given that close contact increases the risk of transmitting infectious diseases [3,4]. Modern slit lamps frequently come with a camera, whether dedicated or integrated into a smartphone, allowing them to capture images and videos of the eye during examinations. This feature is especially beneficial for data collection, telemedicine, and education, as captured images and videos can be viewed on a display, stored electronically, and shared remotely [5-7]. Current camera-equipped slit lamps often use a single-channel design, resulting in the loss of stereoscopic information and, consequently, depth perception during image and video capture. Without depth perception, it can be challenging to visualize subtle anatomical variations and perform procedures guided by the slit-lamp view [8].

Mixed reality (MR) technology enables interactive experiences that blend both real and virtual environments by using a head-mounted display (HMD). A HMD is a wearable device designed to fit comfortably over the user’s head, equipped with a stereo camera and individual screens for each eye. Additional sensors are incorporated to accurately track head and eye movements. The MR headset’s see-through function allows users to view the real world with depth perception. By seamlessly blending digital content with the real world, MR facilitates intuitive interaction and manipulation of virtual objects within the user’s actual environment. These advancements make MR a promising tool for applications such as 3D medical data visualization and telemedicine [9,10].

To address the limitations of conventional slit-lamp examinations in remote collaboration and stereoscopic information sharing, we developed the MR-based slit lamp (MR-SLP) that integrates MR technology with a traditional slit lamp. The MR-SLP enables local operators to interact naturally with patients using its see-through function while simultaneously visualizing a stereoscopic diagnostic view; it also allows real-time streaming of the view to remote physicians wearing headsets for immersive participation. This functionality supports collaborative decision-making, as remote specialists can actively guide local operators during examinations, and enables recording of stereoscopic diagnostic images and videos for documentation and education. In this study, we aim to develop and evaluate the MR-SLP system and assess its capability for real-time stereoscopic imaging, remote visualization, and collaborative ophthalmic examination and telemedicine.

Methods

Ethical Considerations

The study was conducted in accordance with the tenets of the Declaration of Helsinki and was approved by the institutional review board (IRB) of Florida International University (IRB-19‐0112-CR02 and IRB-26‐0004). Written informed consent was obtained from all participants prior to enrollment. All data were deidentified before analysis, and no personally identifiable information or protected health information was collected or stored by the MR-SLP system.

System Design

The MR-SLP system developed in this study was built on a commercial slit lamp (XCEL250, Recheit Inc). Two board-level USB video cameras (ELP-USB16MP01-L75, 1/2.8-inch sensor, maximum resolution: 4656×3496 pixels) formed a stereoscopic pair and were calibrated before mounted on the left and right eyepieces using custom-designed adapters. The adapters allow fine-tuning of the cameras’ positions to ensure accurate alignment of their optical axes with the eyepieces. A schematic of the system is shown in Figure 1.

‎

The videos captured by the 2 cameras were first streamed via USB to a local computer, where they were processed in real time using Open Broadcaster Software Studio (64-bit, version 31.0), an open-source video streaming application. For the video processing pipeline in Open Broadcaster Software Studio, we used the NVIDIA NVENC H.264 hardware encoder to achieve efficient, low-latency streaming. The encoder settings were specifically tuned for real-time performance. Key parameters included rate control set to constant bitrate, bitrate at 10 Mbps, keyframe interval set to 0 seconds for automatic management, and preset configured to “fastest” with “ultra low latency” tuning. The stream used the “baseline” profile to maximize compatibility and minimize processing overhead. The processed videos were formatted into side-by-side 3D mode (Figure 2A), a widely used format for conveying stereoscopic information in image processing [11]. Subsequently, the side-by-side 3D video was transmitted to local and remote Web Real-Time Communication (WebRTC) servers using the WebRTC HTTP ingestion protocol, with an output resolution of either 1080p or 720p. For local streaming, a free WebRTC server (OSSRS/SRS v6.0 from Docker Hub) was deployed on a PC, while remote streaming was facilitated by a commercial WebRTC service provided by dolby.io over the internet. The processed streams were broadcasted wirelessly to MR headsets using the WebRTC HTTP egress protocol. The headsets (Meta Quest 3; Meta Platforms Inc) then rendered the stream into a stereoscopic view with their built-in video player for immersive visualization (Figure 2B). An image in red-cyan stereoscopic format is provided (Figure 2C) as an example, which can be viewed using red-cyan anaglyph 3D glasses. A diagram of the broadcasting architecture is shown in Figure 2D.

‎

Spatial Resolution Evaluation

A US Air Force 1951 resolution target (Edmund Optics Inc) was used to evaluate the spatial resolution of the MR-SLP. The target was placed at the imaging plane of the MR-SLP and imaged at 25× optical magnification. Raw images of the resolution target were captured using the MR-SLP’s USB cameras with the built-in camera app on Microsoft Windows 11, at a resolution of 4656 × 3496 pixels, which is the cameras’ maximum resolution. Images displayed in the MR headset were captured through real-time video streaming, using the built-in snapshot tool of the Meta Quest 3 at a resolution of 1080p.

Intensity profiles were generated across the line pairs for selected group–element combinations from the acquired resolution target images using MATLAB (MathWorks Inc). The minimum resolvable line pairs were determined according to the Rayleigh criterion, wherein the central maximum of one pattern coincides with the first minimum of another. Additionally, a subjective evaluation was conducted in which participants compared the minimum readable line pairs observed directly through the eyepieces with those visible in the images captured via camera.

Tube-Threading Test for Evaluating Visuomotor Performance With the MR-SLP

To assess the visuomotor performance of the MR-SLP, a tube-threading test was designed based on the concept of a bead-threading task [12,13], a well-known method for estimating fine visuomotor performance that was critically affected by depth perception. This tube-threading test consisted of 2 sequential tasks. In task 1, the participant was asked to pick up each of the 10 tubes from a container using a tweezer, place the tube at the focal plane of the slit lamp to be seen clearly, and then return the tube to the container. In task 2, the participant was asked to pick up each tube, thread a wire into it under the visual guidance of the slit lamp, and return the tube to the container. The participants were required to complete 2 tasks 5 times under each designed condition shown in Table 1. High-precision tweezers (PL-30; Fisher Scientific) primarily for microscopy applications were used to grab the tubes. The tubes used in this test were made from micropipette tips. Their outer diameter ranged from 0.8 mm to 1.6 mm, and their inner diameter ranged from 0.4 mm to 1.2 mm. The diameter of the wire used in this test was 0.2 mm. The small size of these tubes and the wire, combined with 10× optical magnification of the slit lamp, required precise depth perception to complete the designed tasks.

Five individuals aged between 21 and 61 years (mean 42.2, SD 16.5 years) were recruited for this test. All participants provided written informed consent to participate in the study. Each participant had normal visual function, with a best-corrected visual acuity of 20/20 or better and a minimum stereoacuity of approximately 40 arc seconds, assessed using random-dot stereograms [14]. All participants did not have any history of ophthalmologic surgery, ocular motility disorders, known fine motor impairments, or other physical impairments that could interfere with the task performance. All participants completed a short acclimation session before the main test. This included (1) adjusting the eyepieces and MR headset to achieve a full field of view at test magnification, (2) finding a comfortable position so that their body and hands were stable and at ease while performing the tube-threading test, and (3) practicing the tube-threading task until they felt comfortable.

The time taken for each task under each condition was individually recorded, denoted as $T_{task1}\left(p,c,i\right)$ and $T_{task2}\left(p,c,i\right)$, where p is the participant number, c is the condition ID, and i is the repeat.

From a single repeat, the time to thread the tube $T_{thr}\left(p,c,i\right)$was calculated as

$T_{thr}\left(p,c,i\right)=T_{task2}\left(p,c,i\right)-T_{task1}\left(p,c,i\right)$(1)

The mean of $T_{task1}\left(p,c,i\right)$ and $T_{thr}\left(p,c,i\right)$ from the 5 repeats of a given participant under the same condition were calculated as

${\displaystyle {\displaystyle \begin{array}{l}{\stackrel{-}{T}}_{task1}\left(p,c\right)=\frac{1}{5}\sum _{i=1}^5{T}_{task1}\left(p,c,i\right),\\ {\stackrel{-}{T}}_{thr}\left(p,c\right)=\frac{1}{5}\sum _{i=1}^5{T}_{thr}\left(p,c,i\right)\end{array}}}$(2)

and $T_{task2}\left(p,c,i\right)$ taken for each task under each condition was individually recorded. ${\stackrel{-}{T}}_{task1}\left(p,c\right)$ and ${\stackrel{-}{T}}_{thr}\left(p,c\right)$ for all 4 conditions were collectively analyzed using statistical methods to identify the effects of the test conditions on participants’ performance.

Telemedicine Assessment

A telemedicine assessment was conducted to evaluate the streaming quality, remote usability, and applications of the MR-SLP. Five participants were assigned to different roles across 3 locations in Miami. The local site was at the main campus of Florida International University, where an ophthalmologist with around 10 years of clinical experience operated the MR-SLP and performed test examinations on a volunteer with normal visual function. Stereoscopic slit lamp video was captured and streamed in real time at an output resolution of 1080p. Simultaneously, a second ophthalmologist with more than 20 years of clinical experience watched the stereoscopic video from the MR-SLP using an MR headset at the first remote site, Bascom Palmer Eye Institute at the University of Miami Miller School of Medicine. He communicated in real-time with the MR-SLP operator at the local site, providing expert guidance and diagnostic opinions. In addition, a senior scientist at the first remote site also observed the examination procedure through an MR headset for technical assistance. Finally, a senior scientist at the second remote site, the Florida International University Engineering Center, observed the process using an MR headset. All local and remote participants verbally assessed the system’s technical performance, including image quality and streaming stability. The latency, particularly the round-trip time, was provided by the remote WebRTC platform. Real-time audio communication between the local operator and remote participants was enabled using Zoom (Zoom Communications, Inc).

Data Analysis

Statistical analyses were performed using R (version 4.5.1; R Foundation for Statistical Computing). The normality of the tube-threading task completion times was assessed using the Shapiro-Wilk test. Given the nonnormal distribution of the data, the nonparametric Kruskal-Wallis test was used to compare conditions, followed by post hoc analysis using the Dunn test with Bonferroni correction. Statistical significance was set at P<.05.

Results

Image Quality of the MR-SLP: Spatial Resolution

Our objective analysis revealed that both the MR-SLP cameras and MR headset were able to resolve element 5 in group 6 of the resolution target, corresponding to a spatial resolution of 102 line pairs/mm with a line width of approximately 9.8 µm (Figure 3). Additionally, we conducted a subjective evaluation with 3 participants who had a best-corrected visual acuity of 20/20 or better. These participants noted that, although they could discern element 5 in group 6 through the MR headset, they were able to identify up to element 6 in group 6 (114 line pairs/mm) when viewing directly through the slit-lamp eyepieces. All participants noted that the lines appeared slightly clearer when viewed through the traditional slit lamp compared with the MR headset display. This subjective difference may indicate that the fidelity of the current digital imaging pathway, which includes the camera and display, may not match that of direct optical viewing with the human eye.

‎

Tube-Threading Test With the MR-SLP

All participants successfully completed 2 sequential tasks (Figure 4) within a reasonable timeframe (<10 min). Their ${\stackrel{-}{T}}_{task1}\left(p,c\right)$ and ${\stackrel{-}{T}}_{thr}\left(p,c\right)$ records are summarized in Tables 2 and 3, respectively. According to the records, ${\stackrel{-}{T}}_{task1}\left(p,c\right)$ was typically the shortest in the direct view condition (c=4). A Shapiro-Wilk test was conducted with ${\stackrel{-}{T}}_{task1}\left(p,c\right)$ and ${\stackrel{-}{T}}_{thr}\left(p,c\right)$ for each test condition. The test outcome indicated that the assumption of a normal distribution was not applicable in this dataset. Therefore, a nonparametric method, the Kruskal-Wallis test, was used to compare ${\stackrel{-}{T}}_{task1}\left(p,c\right)$ and ${\stackrel{-}{T}}_{thr}\left(p,c\right)$ under different conditions. Please find the whole dataset in Table S1 and Table S2 in Multimedia Appendix 1.

‎

As shown in Figure 4D, the distribution of $T_{task1}\left(p,c,i\right)$ for all 4 test conditions are similar. This observation was confirmed by the result of the Kruskal-Wallis test, indicating that there were no significant differences among ${\stackrel{-}{T}}_{task1}\left(p,c\right)$ for all 4 conditions (n=20; P=.77). According to the trend depicted in Figure 4E, the mean value of ${\stackrel{-}{T}}_{thr}\left(p,c\right)$ for each condition was the shortest in the direct view condition (c=4) and gradually increased from test condition 3 (MR 3D 60 frames per second [fps]) to test condition 2 (MR 3D 30 fps) and then to test condition 1 (MR 2D 60 fps). A statistically significant difference in ${\stackrel{-}{T}}_{thr}\left(p,c\right)$ was observed among the 4 test conditions according to the Kruskal-Wallis test (n=20; P=.04). Subsequent post hoc analysis using Dunn test with Bonferroni correction revealed that only ${\stackrel{-}{T}}_{thr}\left(p,1\right)$ and ${\stackrel{-}{T}}_{thr}\left(p,4\right)$ are statistically significantly different, meaning that a statistically significant difference exists only between test condition 1 (MR 2D 60 FPS) and test condition 4 (direct view; n=5; P=.03). This analysis showed that the performance of the threading task had no significant difference between MR headset working on 3D mode and viewing directly through the eyepieces. The results also suggested that increasing the frame rate will reduce the time taken for the threading task.

Telemedicine Assessment

In the test, as depicted in Figure 5, the MR-SLP system’s ability to simultaneously transmit high-resolution stereoscopic video to both local and remote MR headsets was successfully demonstrated. At the local site, the operator wearing the MR headset could simultaneously see the patient through the HMD’s see-through function and view the stereoscopic examination images immersed in their real-world environment. Meanwhile, remote users connected via the internet could concentrate on the stereoscopic examination images, offering real-time expert guidance and diagnostic input, promoting effective collaboration over distances.

‎

During examination of the volunteer’s eye with the MR-SLP, anatomical details of the anterior segment (Figure 6) were accurately captured and displayed in real time on both local and remote MR headsets. According to the ophthalmologists participating in this trial, they reported that the image quality viewed in their MR headsets closely rivaled that observed through the slit-lamp eyepieces. The real-time video streams in this trial also maintained robust depth perception and exhibited low latency at remote sites, with an average round-trip time of less than 40 milliseconds and video jitter below 20 milliseconds. This allowed for a highly interactive and collaborative examination by the local and remote ophthalmologists. Both ophthalmologists involved in the trial stated that the MR-SLP system compared favorably to traditional slit lamp viewing and highlighted its potential advantages for clinical practice.

Fundus examination using the MR-SLP was also evaluated. Following the conventional procedure, the operator, wearing the MR headset, held an ophthalmic lens (90D; Volk Optical) between the volunteer’s eye and the slit-lamp objective to visualize the retina. An example result is shown in Figure 7. Under a nonmydriatic condition, different regions of the retina were clearly visualized in the headsets by adjusting the slit lamp and the Volk lens. Streaming of the fundus images to the local and remote headsets was processed at the background without interfering with the operator’s performance. In the test, the operator required only a brief additional period to adapt to the hand-eye coordination. When operating from alternative positions, such as an off-axis (lateral) position, maneuvering the slit lamp and Volk lens requires establishing a new hand-eye coordination and required additional practice. However, the operator could select the operating position for fundus examination without compromising overall performance.

‎

‎

Discussion

Principal Findings

We tested the performance and validated the feasibility of the MR-SLP for providing stereoscopic slit-lamp views of the eye, including the anterior segment and the fundus, in real time both locally and remotely. The MR-SLP may provide a new approach for slit-lamp examinations by eliminating the need for continuous close viewing through the eyepieces, which may offer more comfort for both the operator and the patient during examinations and more flexibility in operating the machine. Furthermore, the system enhances the efficiency of slit-lamp training by enabling multiple participants to view real-time, stereoscopic diagnostic images as if viewing them directly through the eyepieces. The remote ophthalmologist wearing the MR headset gains an immersive experience as if operating the slit lamp directly, facilitating stereoscopic teleophthalmology.

Factors Affecting Spatial Resolution

The MR headsets used in this study have a display resolution of approximately 2K per eye, while the cameras have a maximum resolution of 4K. The measured spatial resolution was similar (102 line pairs/mm) for both the images acquired by the MR-SLP cameras and the MR device, suggesting that at the tested magnification (25×) the system’s effective resolution was limited by the slit-lamp optics. It is also possible that the MR device’s rendering algorithms enhance the perceived clarity in the region of interest, compensating partially for the lower display pixel density [15]. However, the participants in the subjective evaluation test noted a difference in perceived clarity between the MR headset and direct eyepiece view. Since this observation was based on anecdotal reports from a small group of participants (n=3), future research should implement a larger psychophysical study to statistically quantify and validate any differences in perceived image quality.

Depth Perception, Effective Resolution, Frame Rate, and Latency

Accurate depth perception through the slit lamp is essential for precise clinical diagnosis and surgical procedures [16]. Our camera adapter and calibration algorithm allow fine-tuning of the camera position to precisely match the slit-lamp view with the digital camera view, ensuring consistent spatial mapping. The tube-threading test suggests that the MR-SLP in 3D mode may offer depth perception comparable to that of viewing through the eyepieces. Under test condition 1, participants reported that the main challenge was aligning the wire with the orifice of the tube without stereoscopic cues. A low frame rate also exacerbated the issue by creating mistiming between visual feedback and motion control, which increased the difficulty of the task. Additionally, while all participants completed a brief acclimation session, the study did not formally assess the learning curve for using the MR-SLP system. However, this preliminary test is valuable for the design of a more definitive, larger-scale validation study.

All the participants stated that the resolution target and tubes appeared clearer when viewed directly through the eyepieces. We believe this was caused by the effective resolution of the cameras used in this study during video streaming. The output frame rate was set at 60 fps only when the camera was operated at a resolution of 720p, which was limited by the camera hardware. This hardware bottleneck resulted in lower resolution of the streamed video than the slit lamp’s optics and the MR headset display could provide. This issue can be resolved by using cameras that provide 1080p or higher video resolutions at a frame rate of 60 fps, which are abundant in today’s market. The results of the tube-threading study, shown in Figure 4, also indicate that a higher frame rate led to better performance. This improvement may be attributed to the reduction of the motion blur at higher frame rates [17]. Future studies should explore how to best balance maintaining high frame rates with achieving higher resolutions, such as 1080p or 4K, for these dynamic procedural tasks. We believe that by introducing higher-quality cameras and more advanced MR headsets, such as the Apple Vision Pro, the video quality of the MR-SLP at both local and remote sites will more closely match viewing through the eyepiece of a slit lamp.

System latency is a crucial factor in real-time applications, such as telemedicine, because delays can significantly impact the local operator’s performance and diminish the quality of remote interactions. Latency can arise from network transmission, data processing, and display rendering [18]. While acceptable latency thresholds for demanding applications such as telesurgery are suggested to be below 100 to 200 milliseconds [19,20], platforms using WebRTC architecture can potentially achieve very low media transmission delays [21]. Remote tests conducted within the same city demonstrated a good performance with an average round-trip time of less than 40 milliseconds. Nevertheless, further evaluation through long-distance remote testing is needed.

MR Headset Challenges

In the current implementation of the MR-SLP system, Meta Quest 3 is used as the MR headset for both the local and remote locations. Quest 3 offers advantages for biomedicine applications, including high-resolution displays and advanced color pass-through that enable detailed 3D visualization of anatomical structures. The see-through function and accurate spatial mapping facilitate natural interaction and situational awareness, allowing users to seamlessly integrate virtual content with the real clinical environment. Additionally, Quest 3’s onboard processing power efficiently converts side-by-side images into immersive 3D views, supporting real-time telemedicine and collaborative workflows. However, the headset’s ergonomics and comfort remain suboptimal, particularly during prolonged use, such as continuous wear for more than 60 minutes. While the display resolution is high, further improvements—such as 4K or higher per eye—would be beneficial for even greater image clarity. We are also investigating alternatives to Quest 3, such as Apple Vision Pro, or augmented reality glasses. Our system’s architecture is compatible with other extended reality platforms such as augmented reality for the local applications and virtual reality for remote physicians.

Limitations and Future Studies

While the technique holds promise for enhancing slit-lamp examinations, teleophthalmology, and medical training, several key limitations of our study need to be acknowledged, such as small sample size for statistical analysis and subjective evaluation based on anecdotal reports.

In this study, we did not evaluate the learning curve for different operators associated with adapting to the MR-SLP view, including potential differences in hand-eye coordination during complex maneuvers. For the evaluation of user experience with the MR-SLP, feedback from participating ophthalmologists was collected through verbal reports rather than using a validated assessment scale. Although this approach provided valuable initial insights for a feasibility study, it did not allow for quantitative analysis of user satisfaction or formal measurement of interrater agreement regarding the system. Future studies should use validated questionnaires to enable a more robust evaluation of the system’s usability and acceptance in clinical settings. Additionally, the camera adapter and software implementation are cost-effective and require minimal modification, allowing for versatile integration with most slit lamps and potentially other binocular microscopes.

Conclusions

This study demonstrated the technical feasibility of the MR-SLP in capturing and transmitting real-time, stereoscopic slit-lamp examination videos to MR headsets. Our tests showed that the system can provide visuomotor performance comparable to direct viewing, and it provided low-latency streams for effective remote collaboration.